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ABSTRACT 

This study examined the extent to which student engagement is associated with a traditional 
assessment of student knowledge. In this study, ETS Business Major Field Test (MFT) scores 
were compared to student’s self-reported survey responses to specific questions on the National 
Survey of Student Engagement (NSSE). Areas of the NSSE survey such as Level of Academic 
Challenge, Active and Collaborative Learning, Enriching Educational Experience, Higher-Order 
Thinking, Integration of Diversity into Coursework were included in the study. Grade Point 
Average (GPA) was also compared to MFT scores and NSSE items. While the sample size was 
small from one institution (41 students), a number of measures of student engagement were 
showing signs of linkages to higher MFT scores. 
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INTRODUCTION 


°r nearly three decades, experiential learning has been discussed with two emerging views 
y/f ( http://www.infed.org/biblio/b-explrn.htm ). The first view believes that experiential learning is the 
type of learning that provides the students with the opportunity to acquire, and then apply, knowledge 
and skills to a current, relevant situation. The second view believes that experiential learning is achieved through a 
reflection of everyday experiences. New terms have emerged over the past decade or two such as active learning or 
collaborative learning but for the purpose of this paper, these three terms are essentially interchangeable. Essentially 
all of these terms support a higher level of thinking and more engaged interaction in the classroom that “involves the 
student participating in a Didaktik triangle interaction between the instructor, fellow students, and the discipline 
material'’ (Van Amburgh, et al, 2007, p. 1). 


Promising studies show relationships between experiential learning and academic performance (Carini et 
al, 2006; Busseri & Rose-Krasnor, 2008;), student success (McClenney, 2007), higher-order thinking (HOTS) 
(Carini, Kuh & Klein, 2006; Ives & Obenchain, 2006;) and even emotional intelligence (Manring 2004). An 
impressive study done by Carini, et al (2006) studied the relationships between student engagement and academic 
performance. The study revealed “that levels of student engagement were often positively related to GPA” (p. 13). 
In addition, Carini, et al (2006) reported some small, but significant correlations between student engagement scales, 
RAND, GRE, and GPA measures and self-reported outcomes. Interesting enough, first-year students showed 
correlations between RAND scores and NSSE items such as the number of papers of fewer than 5 pages, coming to 
class having completed readings and assignments, quality of relationships with both faculty and administrative 
offices, and working harder than they thought to meet the instructor’s expectations (Carini, et al, 2006). They 
further found that seniors, however, “benefitted more from working with other students on projects during class, 
integrating ideas from different courses, receiving high quality academic advising, and being at institutions that 
emphasize contact among students of different backgrounds, as well as attendance of campus events and activities” 
(p. 15). Ives and Obenchain’s (2006) study showed that “students engaged in curriculum that emphasizes student- 
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directedness and complex problem solving over focused practice on lower level fact and skill acquisition show a 
significant advantage in HOTS with no loss in LOTS” fp. 72). This could change the emphasis from tests to more 
application-driven work in upper level business courses. 

The importance of application-driven coursework is supported by Kuh (2001) who noted that “state 
legislatures, accreditors, parents, employers, and others want to know what students are learning and what they can 
do” (p. 1). In addition, the focus is shifting from a ranking system, which is sometimes based on an institution’s 
resources and reputation, to an understanding of how the students actually takes the resources that the institution 
provides and uses them for higher levels of learning and personal development. This type of information is relevant 
to institutions of higher learning. As the student progress from their freshman to their senior years, outcomes and 
assignments should be moving from lower level order processes (knowledge and comprehension) to those higher 
order processes (synthesis and evaluation) as described in Bloom’s Taxonomy. McClenney (2007) stated that 
“engagement must be fostered through the intentional design of syllabi, in- and out-of class assignments, 
assessments, and other educational experiences” (p. 143). 

METHODOLOGY 

For the past several years, students taking an upper level capstone class in the college of business have 
taken the MFT. The MFT is an accepted test of core business knowledge and valuable for assessment purposes. In 
addition to the MFT administered in the college of business, the institution also participated in the NSSE survey in 
spring 2007. In the spring of 2009, upper-level business students took both the MFT and abbreviated NSSE survey 
(Appendix). Table 1 shows the student engagement categories and the questions from those categories that were on 
the survey. 


Table 1: Student Engagement Categories and Questions 


Student engagement 
categories (NSSE) 

Questions 

Level of Academic 

Challenge 

• The number of written papers or projects of 20 pages or more during the 2008-2009 
academic year. 

• The number of papers or projects between 5-19 pages during the 2008-2009 academic year. 

• During the 2008-2009 academic year, the extent coursework emphasized applying theories 
and concepts to practical problems or in new situations. 

Active and Collaborative 
Learning 

• How many courses in the 2008-2009 academic year required a presentation? 

• How many courses required working with other students on projects during class in the 2008- 
2009 academic year? 

• How many courses in the 2008-2009 academic year did you participate in community-based 
project as a part of a regular course ? 

Enriching Educational 
Experiences 

• Do you participate in community service or volunteer work? 

• Co-curricular activities you have done in the past year 

• Number of hours per week (on average) that you participate in co-curricular activities? 

• Have you done a study abroad? 

• Have you completed or are currently doing an internship? 

• Worked off campus (# of hours) 

• Have you held a position of leadership on campus (student government, resident assistant, 
club officer, etc.) 

Integration of Diversity into 
Coursework 

• The number of courses in the 2008-2009 academic year that expected you to put together 
ideas or concepts from different courses when completing assignments or during class 
discussions. 

• During the 2008-2009 academic year, the extent coursework emphasized synthesizing and 
organizing ideas, information, or experiences into new, more complex interpretations and 
relationships 

• During the 2008-2009 academic year, the extent that a paper or project that required 
integrating ideas or information from various sources. 
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The researchers added or modified four questions. We added a question regarding the types of co- 
curricular activities the student had done in the past year and the average number of hours worked off-campus in the 
2008-2009 academic year. On the question regarding the number of hours per week the student participated in co- 
curricular activities, we modified the question to exclude intercollegiate sports in order to capture more volunteer 
activities (such as joining a club, being a resident assistant, community service, etc.). Finally, we asked the students 
if they had held a position of leadership on campus. Demographic data regarding gender and rank was also 
gathered. For accuracy purposes, we pulled GPA data from the institution’s database. 

Only the students who completed the MFT and the student engagement survey were included in the 
analysis. This gave us 41 participants. 

All the categorical responses related to the question items of student engagement (except gender, class, and 
metric data) are standardized and converted into metric variables. All standardized scores of the items of student 
engagement are grouped and summated into four categories of student engagement following the NSSE 
classification. Correlation analyses were used to investigate how student engagement items are related to traditional 
measurement of academic performance (GPA) and core business knowledge (MFT). 

RESULTS 

Table 2 shows the demographic characteristics of the respondents and the frequency responses to the 
categories in the student engagement scale I, II, III, and IV. 


Table 2: Brief Description of the Results of Student Engagement Survey (N = 41) 

Question number from survey 

17. Gender: Female (48.8%), Male (51.2%) 

18. Rank: Sophomore (0%), Junior (34.1%), Senior (65.9%) 

19. GPA 

2.66 or lower (2.4%) 

2.67 to 2.99 (14.6%) 

3.00 to 3.33 (14.6%) 

3.34 to 3.66 (41.5%) 

3.67 or higher (26.8%) 

20. MFT (Major Field Test Score) 

135 or lower (4.9%) 

136 to 150 (48.8%) 

151 to 165 (36.6%) 

166 to 180 (4.9%) 

181 or higher (4.9%) 

I. Level of Academic Challenge (Chronbach’s = .547) 

Question 

1. The number of written papers or projects of 20 pages or more during the 2008-2009 academic year. 

0(53.7%) 1-2(39.0%) 3-4 (4.9%) 5 or more (2.4%) 

2. The number of written papers or projects between 5 and 19 pages during the 2008-2009 academic year. 

0(2.4%) 1-2(19.5%) 3-4(43.9%) 5 or more (34.1%) 

3. During the 2008-2009 academic year, the extent coursework emphasized applying theories or concepts to practical 
problems or in new situations. 

Never (0%) Sometimes (39.0%) Most of the time (58.5%) Always (2.4%) 
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4. How any courses in the 2008-2009 academic year required a class presentation? 

0(0%) 1-2(12.2%) 3-4(39.0%) 5 or more (48.8%) 

5. How many courses required working with other students on projects during class in the 2008-2009 academic year? 

0(0%) 1-2(17.1%) 3-4(43.9%) 5 or more (39%) 

6. How many courses in the 2008-2009 academic year did you participate in a community-based project as part of the 
course? 

0(39.0%) 1-2(43.9%) 3-4(12.2%) 5 or more (4.9%) 

III. Enriching Educational Exneriences (Chronbach’s = .520) 

Question 

7. Do you participate in community service or volunteer work? 

Yes (53.7%) No (46.3%) 

8. Co-curricular activities you have done in the past year, (check all that apply) 


Community service and volunteering 

Yes (39.0%) 

No (61.0%) 

School based clubs or groups (except fraternity/sorority) 

Yes (78.0%) 

No (22.0%) 

Community based clubs or groups 

Yes (12.2%) 

No (87.8%) 

Intercollegiate sports 

Yes (41.5%) 

No (58.5%) 

Creative or performing arts 

Yes (2.4%) 

No (97.6%) 

On campus work (student assistant) 

Yes (31.7%) 

No (68.3%) 

Campus publication 

Yes (2.4%) 

No (97.6%) 

Fraternity or sorority 

Yes (2.4%) 

No (97.6%) 

Resident assistant 

Yes (0%) 

No (100.0%) 

Orientation / admissions ambassador 

Yes (4.9%) 

No (95.1%) 


9. Number of hours per week (on average) that you participate in co-curricular activities (organizations, campus 

publications, student government, social fraternity or sorority, etc.). Do not include intercollegiate sports. 

0(14.6%) 1-3(61.0%) 4-7 (9.8%) 8 or more (14.6%) 

13. Have you completed or are currently doing an internship? 

Yes (34.1%) No (65.9%) 

14. Worked off campus (# of hours) 

0(48.8%) 1-5 (4.9%) 6-10(9.8%) 11-20(17.1%) 21 or more (19.5%) 

15. Have you done a study abroad? 

Yes (9.8%) No (90.2%) 

16. Have you held a position of leadership on campus (student government, resident assistant, club officer, etc.)? 

Yes (36.6%) No (63.4%) 
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10. The number of courses in the 2008-2009 academic year that expected you to put together ideas or concepts from 
different courses when completing assignments or during class discussions 

0(2.4%) 1-2(31.7%) 3-4(46.3%) 5 or more (19.5%) 

11. During the 2008-2009 academic year, the extent coursework emphasized synthesizing and organizing ideas, 
information, or experiences into new, more complex interpretations and relationships. 

Never (0%) Sometimes (51.2%) Most of the time (39.0%) Always (9.8%) 

12. During the 2008-2009 academic year, the extent that a paper or project that required integrating ideas or information 
from various sources. 

Never (0%) Sometimes (12.2%) Most of the time (58.5%) Always (29.3%) 

The numbers in parentheses represent the relative frequencies of the responses. Cronbach's a is a measure of the internal 

consistency reliability of an instrument. It measures how well a set of variables or items measures a single construct being 

investigated. The reliability increases as the value of Cronbach's a gets closer to 1. All question items are appropriately classified 
into the four categories which they are supposed to measure. 


In particular, question 4 (class presentation), question 5 (working with other students on projects) and 
question 6 (participating in a community based project as part of a class) seem to be highly related to “Active and 
Collaborative Learning” (.706).” This supports the work of Amburgh (2007), et al and Ostrander (2004) regarding 
the students being more active in class with fellow students, the professor, and their community. In addition, 
institutions have recently encouraged faculty to integrate participation in community based projects into their 
courses. Students gain an understanding of the issues facing the community and learn how they can positively 
impact the community. Ostrander (2004) agreed that students who participate in these community based projects 
have “ ...positive impacts on immediate real-world issues and longer-run concerns about democracy and civic 
participation overall” (p. 88). 

Table 3 shows the correlations between the student engagement items and MFT and GPA. In general, 
while several correlation coefficients are found statistically significant, many of the correlations were not 
significant. In Section A, all marginal correlation coefficients of GPA with student engagement categories are 
negative and three of the four partial correlations of GPA with student engagement categories remain negative even 
after controlling for gender, class, MFT, and other student engagement categories. Section A also indicates that 
MFT is more closely related to student engagement than GPA is because all estimated coefficients of the student 
engagement categories are consistently positive. 

Also, contrary to our expectations, GPA and MFT are inversely correlated with some student engagement 
items (Section B). Preliminary analysis in Section B reveals many items of each scale of student engagement show 
negative relationships with MFT and GPA contrary to our expectation. It may be due to small sample size or the 
validity of MFT and GPA as a measurement of student engagement. Also, the partial correlations (3 ld and 4 th 
column of Section B) indicate that MFT appears to be a slightly better measurement of student engagement than 
GPA because MFT is positively related with more items than GPA. 

In the Enriching Educational Experience category, we added the co-curricular (question 8), number of 
hours per week participated in co-curricular activities - excluding sports (question 9), working off-campus (question 
14) and held a position of leadership on campus (question 16). We believed these questions were a good fit for this 
category. Contrary to our expectations, we found that the more diversity-related content that is integrated into 
coursework, the lower GPA (/;<().05). MFT shows negative marginal (bivariate) relationship with some student 
engagement categories (Level of Academic Challenge and Integration of Diversity into Coursework). However, it is 
very interesting that the relationships between MFT and all student engagement categories dramatically turn to 
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positive sign when they are controlled for all other variables. Based on our preliminary findings, it appears that 
MFT is a more appropriate measurement of student engagement than GPA. This makes sense since GPA is widely 
controversial due to the potential for grade inflation. Finally, a strong positive correlation was found between MFT 
and GPA (.597). As stated previously, this supports the work done by Carini, et al (2006). 


Table 3: Correlations between Student Engagement Items and Major Field Test (MFT) and GPA 


Marginal Correlation Partial Correlation 1 


Section A: Scales of Student Engagement 2 

MFT 

GPA 

MFT 

GPA 

I. Level of Academic Challenge 

-.126 

-.34 ** 

.036 

-.260 

II. Active and Collaborative Learning 

.014 

-.08 

.126 

-.202 

III. Enriching Educational Experiences 

.148 

-.082 

.147 

.057 

IV. Integration of Diversity into Coursework 

-.09 

-.287 * 

.103 

-.398 ** 


Ssection B: Items of Student Engagement 

I. Level of Academic Challenge: 

Marginal Correlation 

MFT GPA 

Partial Correlation 

MFT GPA 

1 .The number of written papers or projects of 20 pages or more during 

the 2008-2009 academic year 

-.190 

-.392 ** 

-.204 

-.108 

2.Thc number of written papers or projects between 5 and 19 pages during 

the 2008-2009 academic year. 

-.031 

-.130 

-.092 

-.152 

3.During the 2008-2009 academic year, the extent coursework emphasized 

applying theories or concepts to practical problems or in new situations. 

II. Active and Collaborative Learning: 

-.052 

-.216 

.287 

-.018 

4.How any courses in the 2008-2009 academic year required a class presentation? 

-.104 

-.142 

.132 

.150 

5.How many courses required working with other students on projects during 

class in the 2008-2009 academic year? 

.183 

.102 

.069 

-.035 

6.How many courses in the 2008-2009 academic year did you participate in 

a community-based project as part of the course? 

III. Enriching Educational Experiences: 

-.046 

-.149 

.024 

-.032 

7.Do you participate in community service or volunteer work? 

-.053 

-.067 

-.136 

.360 * 

8.Co-curricular activities you have done in the past year, (check all that apply) 

.037 

-.282 * 

-.077 

-.406 * 

9,Number of hours per week (on average) that you participate in co-curricular 

activities (organizations, campus publications, student government, social 

fraternity or sorority, etc.). Do not include intercollegiate sports. 

.234 

-.073 

.227 

.231 

13.Have you completed or are currently doing an internship? 

.295 * 

.191 

.221 

.106 

14.Worked off campus (# of hours) 

-.036 

.109 

.061 

-.108 

15.Have you done a study abroad? 

-.142 

-.266 * 

.050 

-.041 

16. Have you held a position of leadership on campus (student government, 

.191 

.097 

.125 

.176 
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Marginal Correlation 

Partial Correlation 

IV. Integration of Diversity into Coursework: 

MFT 

GPA 

MFT 

GPA 

lO.The number of courses in the 2008-2009 academic year that expected you 

to put together ideas or concepts from different courses when completing 

assignments or during class discussions. 

.140 

-.145 

.296 

-.386 * 

11.During the 2008-2009 academic year, the extent coursework emphasized 

synthesizing and organizing ideas, information, or experiences into new, 

more complex interpretations and relationships. 

-.377 ** 

. 427 *** 

-.319 

-.385 * 

12.During the 2008-2009 academic year, the extent that a paper or project 

that required integrating ideas or information from various sources. 

.039 

-.059 

.270 

-.090 


19. GPA 


529 *** 


.361 * 


*, **, *** indicates significance at 10%,5%, and 1%, respectively. 

1) Partical correlation coefficiants are controlled for student class, gender, and all other scales or items. 

2) The answers to all survey items are converted into standardized scores. Then, the scores are summated for each scale of student engagement. 


In addition to the correlation, the data were analyzed using Ordinary Least Squares (OLS) regression. We 
did not report the results since the signs of the coefficients of the OLS regressions are the same as those of the 
partial correlation coefficients obtained in the regression analysis and MFT is again more closely related to student 
engagement than GPA. The OLS did reveal some additional interesting results. For example, seniors tend to get a 
higher score on MFT than juniors when all other variables being equal. Also, male students tend to outperform 
female students in MFT when they are equal in student engagement, class, and GPA. Regression of GPA on the 
categories and personal data shows that female students’ GPAs are significantly higher than those of male students 
when their GPAs are controlled for all other variables. Also, seniors’ GPAs tend to be lower than juniors’ GPAs 
when others being equal. 

From the results of the correlation and regression analysis, the categories of Active and Collaborative 
Learning and Enrich Educational Experiences seem to be related to the students’ performance in MFT because most 
coefficients of the items for the two student engagement categories are positive. However, many coefficients 
estimated from the regression of GPA on the items of student engagement are negative and significant. In other 
words, GPA performance is inversely related to students’ engagement. For example, the data indicate that GPA 
would likely be lower as the student devotes time and energy into “educational experience” (e.g., worked off 
campus, study abroad, participate in community services, etc) and more diversity is integrated into coursework. 

CONCLUSION 

Based on our review of the literature, we did not find any other study comparing the MFT to student 
engagement so this study is breaking new ground. The key finding from this research is once all other variables 
were controlled, the relationships between MFT and all student engagement categories turned positive. While these 
relationships were not significant, increasing the sample size may show a moderate or even strong relationship. The 
data also showed the MFT is a better indicator of student engagement than GPA. 

The results of this study are preliminary and we intend to expand our sample size by sampling additional 
students at our institution as well as expanding the sample to neighboring institutions. This information does, 
however, support the research of others regarding the relationship of learning outcomes (MFT) and academic 
performance (GPA). Although the relationships between learning outcomes and engagement were not as robust as 
we had hoped, it has sparked our interest to learn more. 
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APPENDIX 

This data will be used in aggregate form only to understand if certain behaviors lead to higher ETS Business Major 
Field Test Scores. Your individual responses will be kept confidential. Your participation in this survey implies 
consent to use the aggregated data. 

Please circle the answer that best fits your experience 

1. The number of written papers or projects of 20 pages or more during the 2008-2009 academic year. 

0 1-2 3-4 5 or more 

2. The number of written papers or projects between 5 and 19 pages during the 2008-2009 academic year. 

0 1-2 3-4 5 or more 

3. During the 2008-2009 academic year, the extent course work emphasized applying theories or concepts to 
practical problems or in new situations. 

Never Sometimes Most of the time Always 

4. How many courses in the 2008-2009 academic year required a class presentation? 

0 1-2 3-4 5 or more 

5. How many courses required working with other students on projects during class in the 2008-2009 
academic year? 

0 1-2 3-4 5 or more 

6. How many courses in the 2008-2009 academic year did you participate in a community-based project as 
part of the course? 

0 1-2 3-4 5 or more 

7. Have you participated in community service or volunteer work in the 2008-2009 academic year? 

Yes No 

8. Co-curricular activities you have done in the past year, (check all that apply) 

□ Community service and volunteering 

□ School based clubs or groups (except fraternity/sorority) 

□ Community based clubs or groups 

□ Intercollegiate sports 

□ Creative or performing arts 

□ On campus work (student assistant) 

□ Campus publication 
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□ Fraternity or sorority 

□ Resident assistant 

□ Orientation / admissions ambassador 

□ Other (please specify)_ 

□ None 

9. Number of hours per week (on average) that you participate in co-curricular activities (organizations, 
campus publications, student government, social fraternity or sorority, etc.). Do not include intercollegiate 
sports. 

0 1-3 4-7 8 or more 

10. The number of courses in the 2008-2009 academic year that expected you to put together ideas or concepts 
from different courses when completing assignments or during class discussions 

0 1-2 3-4 5 or more 

11. During the 2008-2009 academic year, the extent coursework emphasized synthesizing and organizing 
ideas, information, or experiences into new, more complex interpretations and relationships. 

Never Sometimes Most of the time Always 

12. During the 2008-2009 academic year, the extent that a paper or project that required integrating ideas or 
information from various sources. 

Never Sometimes Most of the time Always 

13. Have you completed or are currently doing an internship? 

Yes No 

14. Work off campus (average # of hours per week ) in the 2008-2009 academic year 

0 1-5 6-10 11-20 21 or more 

15. Have you done a study abroad? 

Yes No 

16. Have you held a position of leadership on campus (student government, resident assistant, club officer, 
etc.)? 

Yes No 

If yes, please list all of them 


17. 

Please circle one: 

Female 

Male 


18. 

Please circle one: 

Sophomore 

Junior 

Senior 
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